Tool Mentor: Evaluating Code Coverage using Rational PureCoverageÖ
Purpose
This tool mentor provides an overview of how to use Rational PureCoverage to quickly and easily identify the gaps in your testing of Visual C/C++, Visual Basic, and Java programs. To learn more about PureCoverage, including how to filter coverage data, merge and compare runs, and fine-tune data collection, read Getting Ahead with Rational PureCoverage. For step-by-step information about using PureCoverage, see the PureCoverage online Help.
Related Rational Unified Process activity: section "Execute Unit Test" in Activity: Perform Unit Tests.
Overview
PureCoverage collects a detailed set of coverage data for your program and every component it uses. With this data, you can have confidence that the program you're testing has been exercised ù every line, every function, procedure, or method ù before you release it to your customers. To assess the effectiveness of your code coverage, and pinpoint the parts of your code that are not exercised:
Tool Steps
The first step in monitoring coverage is to collect the code-coverage data.
For Visual C++, instrument and run the program, either directly from Microsoft Developer Studio using the PureCoverage integration, or from PureCoverage. PureCoverage instruments copies of the executable and its associated modules. PureCoverage also inserts additional code to detect which functions and lines were executed in the program. PureCoverage shows you its progress as it instruments files.
For Visual Basic, run Visual Basic projects or p-code programs (Visual Basic 6.0 only), or Visual Basic native-code programs (Visual Basic 5.0 or later), either directly from Microsoft Visual Basic using the PureCoverage integration, or from PureCoverage. When you monitor projects or p-code programs, PureCoverage puts the Visual Basic for Applications (VBA) interpreter engine into a special mode that enables PureCoverage to monitor the engine's operation and directly collect coverage data as your code runs. For native-code programs, PureCoverage instruments the program and then collects code-coverage data. PureCoverage instruments copies of the executable and its associated object libraries. PureCoverage also inserts additional code to detect which procedures and lines were executed in the program. PureCoverage shows you its progress as it instruments files.
For Java, run Java applets, class files, or code launched by container programs from PureCoverage (using the Run Program dialog) or from the command line. When you monitor Java code, PureCoverage puts the Microsoft virtual machine (VM) for Java into a special mode that enables PureCoverage to monitor the VMÆs operation and directly collect code-coverage data as the applet, class file, or code runs.
When PureCoverage starts monitoring, it displays the Run Summary window so you can see both numerical and graphical information reflecting the progress of your testing, and check other information about the run. As you exercise your code, PureCoverage records data about its code coverage. You can pause and resume data recording at any time, enabling you to check specific portions of code. You can also take a snapshot of the current data, enabling you to examine code coverage in stages.
When you exit the program, PureCoverage displays the coverage data it collected. Because there can be a lot of data, PureCoverage uses filters to automatically filter out non-critical data from system libraries and other modules before it displays the program's coverage details. As you analyze the coverage data, you can display more or less data.
Tip: In addition to using PureCoverage interactively, you can also incorporate PureCoverage into your test scripts for automated testing. For more information, look up scripts in the PureCoverage online Help index.
More information? Look up developer studio, visual C++, visual basic, java, run summary, snapshots, and filters in the PureCoverage online Help index.
The second step in monitoring coverage is to analyze the code-coverage data that PureCoverage collected.
When you exit the program for which PureCoverage has been collecting data, PureCoverage displays the Coverage Browser window, displaying information such as the number of functions, procedures, or methods (collectively referred to here as functions) hit and missed by your tests, and the number of lines hit and missed by your tests. In addition, you can see the percentage of functions and lines hit, and the number of times each function was called.
You can use PureCoverage's other windows to further examine the program's code coverage. To review coverage data for all functions in the current run, and to sort them by various criteria, use the Function List window. If debug line information is available, and you monitored functions at line level, you can use the Annotated Source window to analyze a specific function's coverage line by line. Line-by-line coverage data is not available when you monitor a Visual Basic 6.0 p-code program. For the most detailed coverage data, monitor your Visual Basic 6.0 project directly from within Visual Basic 6.0 using the PureCoverage integration.
PureCoverage provides several ways to display only the data you're interested in. For example, you can filter out data by module name, source file name, or function name. PureCoverage automatically filters out non-critical data from system libraries and other modules before it displays the program's coverage details. As you analyze the coverage data, you can display more or less data. You can also use the Show/Hide Columns dialog to display only the information you are currently interested in.
By default, PureCoverage creates an Auto Merge entry in the Navigator window, so you can easily analyze the total coverage of the program over multiple runs. You can also merge specific runs yourself, to analyze coverage.
More information? Look up coverage browser window, function list window, annotated source window, filters, merging runs, and debug data in the PureCoverage online Help index.
The third and final step in monitoring coverage is to rerun your tests, to see whether changes have caused the coverage to improve or regress.
After you make changes to your code, you can rerun the updated program and compare the new results to the previous run.
Use the Navigator window to keep track of all the runs you're working with. You can save coverage data as a PureCoverage data file (.cfy), to use for further analysis or to share with other PureCoverage users. You can save data to a tab-delimited ASCII text file (.txt) to use outside of PureCoverage, for example, in test scripts or in Microsoft Excel. You can also copy data from the Coverage Browser window to use in a text editor or from the Function List window to perform further analysis in Excel.
To track your code coverage from release to release, archive your saved .cfy files with your product releases.
Tip: If you have incorporated PureCoverage into your test scripts, you can automatically save coverage data to a text file (.txt). For more information, look up scripts in the PureCoverage online Help index.
More information? Look up navigator window and saving data in the PureCoverage online Help index.
